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Abstract 

The different orthogonal relationships that exists in the Lowdin orthogonalizations is presented. Other 
orthogonalization techniques such as polar decomposition (PD), principal component analysis (PC A) and 
reduced singular value decomposition (SVD) can be derived from Lowdin methods. It is analytically 
shown that the polar decomposition is presented in the symmetric orthogonalization; principal compo- 
nent analysis and singular value decomposition are in the canonical orthogonalization. The canonical 
orthogonalization can be brought in into the form of reduced SVD or vicc-vcrsa. The analytic relation 
between symmetric and canonical orthogonalization methods is established. The inter-relationship be- 
tween symmetric orthogonalization and singular value decomposition is presented. 



Keywords: Canonical orthogonalization, polar decomposition, principal component analysis, reduced 
singular value decomposition, symmetric orthogonalization. 
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Introduction 



Orthogonalization methods come in two categories, viz., sequential and democratic. The Gram- 
Schmidt method pQ takes the linearly independent set of vectors one-by-one and gives an orthonormal 
set. The democratic orthogonalization methods due to Lowdin namely the symmetric and canonical 
orthogonalizations handle all the given vectors simultaneously and treat them on equal footing. Lowdin 
orthogonalization methods [2J 0] were discovered for the purpose of orthogonalizing hybrid electron orbits 
in quantum chemistry. A few other orthogonalization methods have been developed independently to 
deal with specific problems in computer science, mathematics, statistics and biology etc. The centrality 
of Lowdin orthogonalization schemes to other orthogonalization techniques such as polar decomposition, 
principal component analysis and singular value decomposition is presented. The derivation and inter- 
esting geometric properties of the two procedures are described in the references [U \5\ [5] ■ Applications of 
Lowdin methods to a variety of interdisciplinary problems have been explored in the last decade. They 
have applications in cognitive phenomena [7j [8] , data reduction [9] and in the generation of new polyno- 
mials [10] . 

Lowdin Orthogonalizations 

Let V = {vi,V2, ■ ■ • ,vn} be a a set of linearly independent vectors in a A-dimensional space which 
can in general be a complex vector space. A general non-singular linear transformation A can transform 
the basis V to a new basis Z: 



Z = 



VA. 



(1) 



The set Z(= {z K }) will be orthonormal if 



< Z|Z >=< VA|VA >= A f < V|V > A = A* MA = I, 



(2) 



where M is a Hermitian metric matrix of the given basis V. 



A general solution to the orthogonalization problem is obtained using the substitution 



A = M 1/2 B, 



(3) 



where B is an arbitrary unitary matrix. 



The specific choice B = I gives the symmetric orthogonalization, 



Z = * = V1VT 1/2 



(4) 



while B = U, where U diagonalizes M gives the canonical orthogonalization, 



Z = A = VUd~ 1/2 . 



(5) 
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These two orthogonalized basis sets have novel geometric properties [5] when we consider the Schwcin- 
ler - Wigner matrix [6] in terms of the sum of squared projections. 

Polar Decomposition 

The polar decomposition [TT] of a matrix V S M. nxN , where R is a set of real numbers, can be 
obtained from the Lowdin's symmetric orthogonalization. The symmetric orthonormal basis of a matrix 
V is expressed as 

* = VrVr 1/2 , (6) 
where <fr € ^nxm an( j jyj e pxm^ multiplying both sides of eqn. © from right by M 1 / 2 , we get 

or,V = #M 1/2 . (7) 
This is called as polar decomposition of the matrix V. 

Principal Component Analysis 

Let V G M. Nxn , then the sum of squares and cross-prodcuts (SSCP) matrix S can be constructed 
using VV T . The SSCP matrix is the covariance matrix without subtracting the mean. The diagonal 
values are sums of squares and the off-diagonal values are sums of cross products. The eigenvalues and 
eigenvectors of the SSCP matrix S are constructed. We have found that the eigenvalues are the same as 
the eigenvalues of the Gram matrix constructed using V T V in the case of Lowdin orthogonalizations. 
The eigenvectors in the two cases are different. However, the eigenvectors of the SSCP matrix S, called as 
the principal components |12j of V, are the same as those obtained using the canonical orthogonalized set 
A = VUd -1 / 2 . The eigenvectors of S are ordered so that the first two principal components retain most 
of the variance present in the original set of vectors. We have computed the sum of the projection-squares 
[5] of the given vectors onto principal components [13] . This gives the eigenvalues of S and is the same 
as the sum of projection-squares of the original vectors on the canonical orthonormal vectors, i.e., the 
eigenvalues of M. 

For N vectors in n dimensions, we find that the principal components obtained through the principal 
component analysis of a square matrix V are equal to the orthonormal vectors obtained through the 
Lowdin's canonical orthogonalization. Hence in the case of square matrices, the principal component 
analysis of the pure SSCP matrix is equivalent to the canonical orthogonalization. 
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Singular Value Decomposition 



The singular value decomposition |14[ I15j of a non-singular matrix V can be obtained from the 
Lowdin's canonical orthogonalization. The canonical orthogonalization of a matrix V can be written as 

A = VUd" 1/2 . (8) 

Multiplying both sides of equation (JSJ from its right with d 1 / 2 , we have 

Ad 1/2 = VU I = VU. (9) 

Now multiplying equation ([9]) on both sides from right with , we get 

V = Ad 1 / 2 ^ (10) 

This is the singular value decomposition of matrix V and is called as reduced singular value decomposi- 
tion form of the canonical orthogonalization. 

Analytic Relations between Symmetric and Canonical Orthogonalizations 

We can analytically obtain the relationship between symmetric and canonical orthogonalizations. If 
one of them is obtained from the Hermitian metric matrix, say canonical or symmetric, then the other 
can be obtained from the following fundamental relations. 

The symmetric orthonormal basis is given by 

* = VM 1/2 . (11) 

From the equation Z = VA = VM X / 2 B and for B = U, We have 

A = V]Vr 1/2 U (12) 
Or, A = *U. (13) 

This analytic relation is useful to construct the canonical orthonormal basis directly using symmetric 
orthonormal basis and the eigenvectors of the Hermitian metric matrix. 

The symmetric orthonormal basis can be obtained from the canonical orthonormal basis by multiplying 
both sides of equation (|13[) from its right with LF as shown below. 



AU f = * UUt (14) 

AU f = #1 = #. (15) 

Or, * = AU f . (16) 
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Symmetric Orthogonalization in SVD 



We can analytically obtain the symmetric orthogonalization from the singular value decomposition. 
Let V £ R nxm with n > m be a non-singular matrix. Then the singular value decomposition of V can 
be written as 



W21 W22 



Win 
W 2r , 



\ 



ai 

(7 2 



\ 



y W n i W n2 ■ ■ ■ W nm J \ ■ ■ 

WSU', where (S = d 1/2 ), 



/ V 



till U12 
"21 u>22 

"ml «to2 



«2n 



\ 



where W £ R nxm anc j XJ £ jj mxm are matrices with their columns as orthonormal vectors. The columns 
Wj, j = 1, 2, ■ • • , m of W are called Ze/t singular vectors of V and the columns of Uj, j = 1, 2, • ■ • ,m of 
U (or rows of TJt)arc called right singular vectors of V. And S £ R mxm i s square and diagonal matrix 
with Cj's as the singular values of V. By convention, the singular values are arranged in a descending 
order as o~\ > 02 > ■ • ■ > tr m > 0. This form of singular value decomposition is known as reduced singular 
value decomposition. This reduced SVD can give the symmetric orthogonalization. Using the analytic 
relationship between the symmetric and canonical orthogonalizations, the symmetric orthogonalization 
of the matrix V can be written from the reduced SVD as follows, 



* = WU f , 

*y = Wil (ut) 1 . + w i2 (ut) 2 . + ... + w im (ut) m . 

= W (1 (U) jX + W l2 (U) . 2 + • • • + W im (U) jm , 



(17) 
(18) 
(19) 



where W is the same as the canonical orthonormal basis A. The symmetric orthonormal basis $ is 
unique since it takes the linearly independent set of columns of V as input and gives the orthonormal 
columns of WU* as output. 



Discussion 



The democratic orthogonalization procedures considers the entire lot of vectors in one go. We con- 
centrated mainly on the democratic type because their applications have not been explored much except 
in the domain of quantum chemistry. We have established that the canonical orthogonalization was in 
fact invented independently by several people from time to time with different names such as princi- 
pal component analysis and singular value decomposition. We have established their equivalence. The 
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connection between symmetric and canonical orthogonalizations have also been established. Analytic re- 
lation between symmetric and canonical orthogonalizations methods is presented. The inter-relationship 
between symmetric and SVD is also shown. 
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